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Abstract 
Prevailing citation practice in law journals is to use uniform resource 
locators (URLs) when citing electronic sources. Digital object identifiers (DOI) 
provide a more reliable and robust mechanism for citing digital, scholarly articles. 
This study examines to what extent DOI names exist but are not used in law 
journal citations. Citations to scholarly articles from twenty- five randomly- 
selected articles appearing in the 2007-2008 volumes of four major law journals 
(Harvard Law Review, Columbia Law Review, Yale Law Journal, and University 
of Pennsylvania Law Review) were checked for existing DOI names using 
CrossRef's Simple Text Query form. This resulted in 394 citations that could have 
had DOI names, but did not. This number suggests that law journal editors and 
librarians should consider adding DOI names to citations. For journals that 
publish exclusively online or are interdisciplinary, assigning DOI names to their 
own articles may be a prudent measure to better ensure long-term digital access 
and citation by scholars in other fields. 
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Introduction 

Citation practice in law journals has a long, venerated history. A key 
function of a citation is enabling researchers to locate the unique resource that is 
being cited. The traditional rules for citing scholarly articles have performed this 
function pretty well — 120 Harv. L. Rev. 393 points to the one article that starts on 
page 393 of the Harvard Law Review's 120 th volume. In addition to being a 
unique identifier for articles, the volume-title-page citation is persistent — it is 
highly unlikely that the hard copy journals would ever be re-paginated such that 
the identifier would no longer reliably represent the cited article. 

However, like many things, digital communications have rendered long- 
standing practices less useful than they used to be. Journals now routinely 
reference electronic sources, including electronic versions of scholarly articles, 
and some journals even publish exclusively online. The traditional citation system 
based on paper does not seamlessly map onto digital articles. When articles are 
not initially published in paper or PDF format, then traditional page numbers do 
not make much sense. Even when originally paper articles are digitized and 
posted online, the most common means of pointing to these documents, the 



Uniform Resource Locator (URL) is unreliable due to "link rot." 1 

Persistent digital identifiers have been created to help ameliorate this 
problem, and one device, the digital object identifier (DOI), has been generally 
adopted by many scholarly publishers. 2 While many publishers in non-legal 
disciplines assign DOI names, law journals have not made much use of DOI 
names in footnotes or as identifiers for their articles. This study looks at a 
snapshot of a selection of articles published in 2007 and 2008 in the law journals 
that edit the Bluebook, the dominant legal citation guide. 

To decide whether legal scholarship should join other disciplines in 
assigning and citing with DOI names, it would be useful to have some idea of the 
extent to which articles cited in law journals have DOI names that could be, but 
are not, included in the footnotes. If enough articles cited in law journals have 
DOI names, then journal editors, law librarians, and producers of legal citation 
guides will have evidence to support using DOI names in citations. 

Literature Review 
Proper citation is important in all serious scholarship, but for legal 
scholarship precise and accurate citation is particularly emphasized. As Rumsey 
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has noted, "Access to the authority underlying (and cited in) the author's work is 
crucial for scholarship." 3 Due to their extensive background discussions and 
research documentation, many law journal articles become references for their 
niche areas of law. Accurate and reliable retrieval of cited resources is thus crucial 
to one of law journals' core functions. 

While traditional citation of paper resources somewhat satisfies this need, 
it fails to permit adequate citation of exclusively digital articles. Some non-legal 
(such as D-Lib Magazine and First Monday) and legal (like the Duke Law 
&Technology Review) digital journals do not use conventional pagination, and 
while many online law review supplements do provide articles in PDF with 
pagination, they are never widely printed, and complete citations require some 
sort of digital identifier (at present, generally a URL). For articles that are 
published both in print and online, providing parallel citations to both media 
increases accessibility. Electronic links, most commonly as URLs, are necessary 
for effective citations in law journals, no matter whether the journal is published 
in print or digital format. 

However, several studies have raised serious concerns about URLs' long- 
term reliability. In 2002 Rumsey studied citations containing URLs from 1997 to 
2001. She found that only 61% of links from 2001 still worked, and that 
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percentage dropped to 30% for links made in 1997. 4 Most of the references using 
URLs were probably not to law journal articles, but instead news stories, 
government documents and organizational publications. Rumsey recommended 
that journals include parallel citations to more stable print and commercial 
database sources in case links break. She also advocated for journals or libraries 
retaining print copies of cited online sources. 5 

Two other studies provided further evidence that link rot is a problem in 
law journals. Neacsu examined twenty articles that cited to URLs and found that 
twelve of the articles had broken links. 6 Canick compared the persistence of 
references using URLs and references to commercial databases. He found that 
resources in commercial databases generally fared better in terms of retrievability 
than sources on the World Wide Web. 7 In 2006 Davis studied a narrower range of 
URLs in Washington state law journals. She found that roughly 40% of links were 
broken and that most URLs were from the .com, .org and .gov domains. 8 Since 
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most law journals are based at law schools and thus would be in the .edu domain, 
it is likely very few of these URLs pointed to law journal articles. While no 
articles were found studying the longevity of URLs for law journal articles, given 
the frequent turnover of journal staff and inevitable revision of law school and 
journal websites, there is little reason to think that article URLs can remain 
functional for the long term without some effort. This is especially bothersome for 
law journals, which, according to a 2006 study, cite other law journal articles in 
almost 60% of their citations. 9 

Given this troubling evidence for link rot in law journals, solutions needed 
to be developed. Proposals to maintain print copies, provide parallel cites to print 
resources in case the URLs break, and citing to commercial databases have been 
mentioned. Lyons suggested adopting persistent digital identifiers to combat link 
rot. These identifiers point to a specific digital document instead of the location 
on a server that is hosting it. Thus, if the document is moved to another server due 
to a website redesign or because a journal is taken over by another publisher, the 
persistent identifier would still allow the document to be retrieved. After 
reviewing the various models for persistent digital identification, she urged 
librarians to assist journals in adding digital object identifiers and persistent URLs 



9 Vincent Lariviere et al., The Place of Serials in Referencing Practices: Comparing Natural 
Sciences and Engineering with Social Sciences and Humanities, 57 J.AM. Soc'YlNFO. Sci. & 
Tech. 997, 1002 (2006) [doi: 10.1002/asi.20349]. 



to footnotes when possible. 10 

There are several models of persistent digital identifiers, but the digital 
object identifier model 11 is most relevant to law journals because it is the system 
used most commonly by scholarly journals. Atypical DOI looks like this: 
10.5555/journal2005.4. The "10." simply marks the string of characters as a DOI, 
and the "5555" is a prefix assigned to a particular publisher. The suffix that 
appears after the forward slash points to a specific scholarly work and can be any 
arbitrary set of characters. Some publishers simply use a string of numbers, while 
others use a combination of numbers, letters, and punctuation marks. When a DOI 
is resolved by adding it to the end of http://dx.doi.org in a web browser or 
entering it into a resolver form, the researcher is sent to the most current location 
of the official version of the scholarly work. The publisher can decide whether the 
article is open access or requires a subscription or fee. Even if a publisher 
rearranges its website, changes names, or is absorbed by another publisher, the 
DOI system maintains a central and current index that enables researchers to find 
articles online, while normal URLs would be broken by such changes. When 
accessed through computers of libraries that subscribe to databases containing 
articles with DOI names, those DOI names, along with an OpenURL resolver 
program, will send researchers to the versions of articles in accessible databases. 



10 Susan Lyons, Persistent Identification of Electronic Documents and the Future of Footnotes, 
97 LawLibr. J. 681 (2005). 

11 Int'l DOI Found., The DOI Handbook, (4.4.1 ed. 2006) [doi: 10.1000/186]. 



Due to these advantages, the DOI has become the standard digital 
identifier for scholarly publishing, with most hard science and many social 
science and humanities publishers using DOI names for their articles. 12 The latest 
edition of the Publication Manual of the American Psychological Association 
recommends including DOI names in citations when possible. 13 Over 40 million 
DOI names have been registered through CrossRef. 14 Law journals, however, 
have yet to generally adopt use of DOI names either by assigning DOI names to 
articles or including DOI names in citations. A recent search of Westlaw's law 
journal database indicated that only 383 articles in the entire database of 
thousands of articles contained any mention of DOI names. This could partly be 
due to the fact that the major citation guides for law journals, the Bluebook 15 and 
the ALWD Citation Manual 16 do not mention DOI names, although both give 
citation examples that use Westlaw and Lexis's unique database identifiers, 
indicating that the editors recognize the importance of unique digital identifiers. 

Law journal citations that only use URLs and commercial database 
identifiers may generally lead researchers to the desired sources. But journals are 
giving up significant benefits by not using DOI names, including more persistent 
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access and links to citing articles. This study seeks to examine how many 
scholarly articles cited in law journals have existing DOI names that could have 
been, but were not, included in the citation. 



Methods 

The Bluebook is edited by four major law journals, the Harvard Law 
Review, Yale Law Journal, Columbia Law Review, and University of Pennsylvania 
Law Review. Since these journals have long been leaders in legal citation practice, 
this study will examine to what extent scholarly articles are cited in their pages 
that could have had DOI names included in their citations. A search was 
performed of Westlaw's database for each journal for "doi 12 10." ("doi" within 
two words of "10.") A common form of citing a DOI looks like this: "DOI: 
10.5555/article," so this search would retrieve articles that included DOI names in 
their footnotes. All four searches retrieved no documents, so none of the four 
B/uefaook-editing journals appear to have included DOI names in their citations. 

All 87 scholarly articles written by non-students from the 2007-2008 
volumes of the Harvard Law Review (volume 121), Yale Law Journal (volume 
117), Columbia Law Review (volume 108), and University of Pennsylvania Law 
Review (volume 156) were downloaded from Westlaw. Non-student works were 
identified by excluding any piece whose author was anonymous or identified as a 
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student or JD candidate. Each article had a unique identifier made by abbreviating 
its Bluebook citation (121HLR394, for example). Twenty-two (25.3%) of the 87 
scholarly articles were selected by a random number generator 
( http://www.randomizer.org/form.htm ). This study focuses on articles, essays, and 
book reviews by professors, jurists, or practitioners because they constitute the 
most common category of article published in law journals. Student- written 
pieces, memorials, or administrative writings that might have different citation 
practices were excluded. The footnotes in each of the selected articles were 
manually examined, and any citations to scholarly articles were copied to an 
Excel spreadsheet. Scholarly articles were identified by being cited as being in a 
consecutively-paginated journal (e.g., 121 Yale L.J. 42, as opposed The Ledger 
May 2006, at 58, the citation form for a non-consecutively-paginated magazine). 
The extracted citations were sorted alphabetically and duplicate citations were 
deleted to prevent frequently cited articles from making the sample appear to have 
more cited articles with DOI names than there were, resulting in 1,041 citations 
from the 22 selected articles. 

These citations were then checked for existing DOI names. CrossRef, the 
DOI registration agency for scholarly publishing, provides an online form that 
examines citations and retrieves DOI names if they exist. 17 The form is a large 
text box into which lists of references can be entered. While a specific citation 



17 CrossRef, Simple Text Query Version 1.3, http://www.crossref.org/SimpleTextQuery/. 



11 



style is not required by the CrossRef form, testing showed that the form did not 
recognize the Bluebook volume-title-page convention or journal title 
abbreviations. Thus, each citation had to be modified by adding the full journal 
title, moving the volume number and starting page number together, and 
removing any signals and pinpoint citations. So, for example, "John Smith, 
Article Title, 121 J. of Pol. 934, 950 (2007)" was changed to "John Smith, Article 
Title, Journal of Politics 121:934 (2007)." These modifications were made to the 
citations in the spreadsheet and the modified citations were copied and pasted into 
a numbered list to comply with the CrossRef form's instructions that reference 
lists either be numbered or in alphabetical order by author's last name. Citations 
were pasted into the form in groups of forty citations to prevent the system from 
timing out and failing to complete retrieval of DOI names. While it is impossible 
to know whether the form failed to retrieve some DOI names that did exist, the 
citation form was tested by entering citations to articles that had been 
independently found to have DOI names, and the form correctly retrieved the DOI 
names. The accuracy of the DOI names retrieved was spot-checked by resolving 
DOI names to see if the correct article was found. While in a few instances the 
form retrieved two DOI names for an article and only one DOI name was correct, 
all the other checked DOI names led to the assigned article. If a DOI was 
retrieved for a citation, the DOI was copied into the spreadsheet. 
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Findings 
Of the 1,041 cited articles, 394 (37.8%) had DOI names, and 647 (62.8%) 
did not. With a few major exceptions, most major law journal articles did not have 
DOI names assigned to them. Table 1 displays the number of DOI names found 
that could have been used, broken down by publisher. JSTOR had the most DOI 
names by far because it is the only source of DOI names for articles appearing in 
major law journals like the Harvard Law Review and Yale Law Journal. The 
University of Chicago Press published the Journal of Legal Studies, another major 
law journal, and several political science journals that were cited in the sample. 

Table 1 — Number of DOI names, by assigner (n=394) 



DOI name assigner 


Number of DOI names 


JSTOR 


264 




University of Chicago Press 


35 




Wiley-Blackwell 


15 


Elsevier 


11 


Cambridge University Press 


10 


Oxford University Press 


9 


Sage 


7 
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DOI name assigner 




Number of DOI names 


Annual Reviews 


6 




University of California Press 


6 


Springer 


4 




Public Library of Science 


4 




American Psychological Association 


3 


Project MUSE 


3 


American Association for the 
Advancement of Science 


3 


Guttmacher Institute 


2 


American Medical Association 


1 




Nature Publishing Group 


1 


Massachusetts Medical Society 


1 


Mary Ann Liebert, Inc. 


1 


BMJ Publishing Group 


1 


Royal College of Psychiatrists 


1 


Duke University Press 


1 
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DOI name assigner Number of DOI names 

American Economic Association 1 

Institute for Operations Research and 1 
the Management Sciences 

Routledge 1 

Now Publishers 1 

World Health Organization 1 

J-STAGE 1 



Table 2 displays the legal journals that had more than ten articles cited in 
footnotes that omitted existing DOI names. The journals from which many 
articles with DOI names were cited are, not surprisingly, highly-regarded law 
journals. More interesting, though, are the most-cited journals that did not have 
any articles with DOI names. 

Table 2: Existing DOI names that were not included in footnotes, 
by law journal with more than ten cited articles 

Law Journal Number of DOI names 

Harvard Law Review 46 
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Law Journal 






Number of DOI names 


Yale Law Journal 




38 




Stanford Law Review 




24 




University of Pennsylvania 
Review 


Law 


19 


Columbia Law Review 




17 




Journal of Legal Studies 


17 


University of Chicago Law Review 


17 


Michigan Law Review 


15 



Table 3 shows the most frequently cited journals, whether or not the journals use 
DOI names. By comparing Table 2 and Table 3, we can see that several of the 
most-cited law journals did not use DOI names. Texas Law Review, New York 
University Law Review, and Georgetown Law Journal are heavily cited journals 
that have do not use DOI names at all. 



Table 3: Ten most cited journals 



Law Journal 


Number of Cited Articles 


Harvard Law Review 


64 
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Yale Law Journal 


53 


Stanford Law Review 


42 


University of Pennsylvania Law Review 


34 


Columbia Law Review 


31 


Texas Law Review 


28 


Michigan Law Review 


25 


New York University Law Review 


22 


University of Chicago Law Review 


21 


Georgetown Law Journal 


20 



While the three most-cited journals (Harvard Law Review, Yale Law 
Journal, and Stanford Law Review) have some DOI names through JSTOR, the 
database's moving wall means that the most recent articles will not have DOI 
names. Many articles are cited when they are most timely, so reliance on JSTOR 
means the benefits of DOI names are forfeited during a time current articles may 
be cited. For the Harvard Law Review, for example, 64 articles were cited, but 46 
had DOI names, so nearly a third of the cited articles from that journal did not 
have DOI names at the time (though they are forthcoming once the embargo 
period ends). 

The law journal articles that did have DOI names fell largely fell into two 
categories: articles published by large academic publishers (such as Oxford 
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University Press or the University of Chicago Press) and articles hosted on 
JSTOR. For example, the Journal of Legal Studies is published by the University 
of Chicago Press and thus, along with all of the Press's other journals, has articles 
with DOI names. Most law journals are not published by academic publishers, but 
rather by law schools, which reduces the number of DOI-assigning journals. The 
journal aggregator JSTOR has begun assigning DOI names to articles in its 
database, 18 and a few major law journals' content, including the four Bluebook- 
editing journals examined in this study, have been archived in JSTOR. Of all the 
journals in Table 2 that had at least ten DOI names that could have been used in 
references to their articles, only the Journal of Legal Studies' articles had DOI 
names that did not resolve to JSTOR. 

Discussion 
This snapshot of references suggests that a fair number of scholarly (legal 
and non-legal) articles cited in law journals have DOI names that are not being 
included in the references. This is unfortunate because DOI names provide 
benefits that traditional Bluebook citations and URLs do not, including greater 
long-term stability and more robust linking between citing and cited articles. 
Under current legal citation standards, law journals are not taking full advantage 
of DOI names. Even if the Bluebook editors never chose to follow the American 



18 Michael Spinella, JSTOR and the Changing Digital Landscape, 36 INTERLENDING & 
Document Supply 79 (2008) [doi: 10.1108/02641610810878549], 
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Psychological Association and recommend DOI names be included in citations, 
journals could modify their in-house citation style by adding DOI names to their 
otherwise B/uebook-compliant footnotes. This would increase the utility of some 
footnotes without causing much confusion to researchers. Journals can either ask 
authors to provide DOI names in their citations or look up the DOI names during 
the process of verifying cited sources. 

Journal staff would have to create reference lists from the footnotes and 
modify the citations to work with the CrossRef tool. While adding this step to 
cite-checking procedures would add some work, once a method for modifying 
Bluebook citations into a style the CrossRef form will accept is settled, it should 
not be too burdensome. Some authors can supply DOI names. Many journals print 
the DOI name on the first page of the article, so cite-checkers can find it in the 
normal course of verifying bibliographic information. Finally, CrossRef has a 
searchable list of journals 19 that use DOI names, enabling staff to reduce the 
number of references that need to be formatted for the CrossRef form. All of these 
factors help journals add DOI names without adding an undue amount of time to 
cite-checking. Journals could also include DOI names while more closely 
following Bluebook style by expressing the DOI name as a URL by adding 
"dx.doi.org/" before the DOI name. In that case, an article with the URL 
"http://www.publisher.com/journal/march_l" and DOI name "10.5555/jrnll234" 



19 CrossRef, Title List, http://www.crossref.org/titleList/. 
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would be cited with the URL "http://dx.doi.org/10.5555/jrnll234." This DOI- 
based URL would be more stable and persistent than the standard URL. 

A further step journals can take is joining CrossRef and assigning DOI 
names to their articles. It has been suggested that DOI fees (an annual CrossRef 
membership fee and one dollar per DOI) 20 may be an obstacle for small 
publishers, 21 and journals will have to carefully consider whether the fees are 
worth the benefits DOI names offer. These identifiers may be particularly used for 
law journals that publish articles exclusively online and journals that are very 
interdisciplinary in scope. Online journals could use DOI names to help avoid link 
rot in article URLs, thereby maintaining more reliable online accessibility over 
the long term. Interdisciplinary journals may benefit from increased access by 
researchers in other fields by using DOI names to give non-legal scholars a 
familiar and convenient means of retrieval. While some legal scholarship is 
valuable to non-legal researchers, lack of familiarity with or access to legal 
databases may present obstacles to research. 22 Assigning DOI names to law 
journal articles can provide non-legal scholars a parallel access method, while 
legal scholars are likely to continue relying on legal databases such as Lexis, 
Westlaw and HeinOnline. While JSTOR appears to have begun this process for 



20 CrossRef, Publisher Fees, http://www.crossref.org/02publishers/20pub_fees.html. 

21 Robert P. Dellavalle, et al, Going, Going, Gone: Lost Internet References, 302 SCIENCE 787 
(2003) [doi: 10.1126/science.l088234]. 

~" Mikhail Koulikov, Indexing and Full-Text Coverage of Law Review Articles in Nonlegal 
Databases: An Initial Study, 102 LawLibr. J. 39, 41-42 (2010). 
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the most elite law journals, journals that are open access, specialized in scope, or 
outside the highest tier will need to adopt DOI names on their own if it is to 
happen in the near foreseeable future. 

This method of selecting articles and citations and finding DOI names is 
far from perfect, and this study does not determine what proportion of law journal 
references generally could include DOI names. Rather, this snapshot simply 
shows that the number of omitted DOI names is not trivial. It is certainly possible 
that errors were made when extracting citations to scholarly articles and DOI 
names, but most mistakes would lead to an existing DOI being missed rather than 
purporting to find a DOI that does not exist. If anything, then, this study under- 
counts the number of DOI names that could have been included in citations. 

Conclusion 
Traditional legal citation has a long history, but new mechanisms are 
needed to most effectively cite and link scholarly legal articles. Digital object 
identifiers provide an established solution that has already been widely accepted 
in other disciplines. DOI names can be added to footnotes without doing much 
violence to Bluebook style or placing inordinate burdens on journal staff. This is a 
very preliminary exploration into the suitability of DOI names for law journals, 
but the non-trivial number of articles cited that had DOI names suggests that 
journal editors, article authors, and academic librarians who work with legal 
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materials should consider further investigation of DOI names and their role in 
further bringing legal citation into the digital age. 



